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ABSTRACT 



Common elements identified as context variables that 



may affect student evaluation of instruction are explored , and the 
literature on student ratings of teacher effectiveness is reviewed. ^ 
Sociocultural context variables (gender , race, ethnicity, and social 
class variables) are discussed in terms of stereotyping and prejudice 
found with non-dominant groups. Dimensions of the context variables 
identified in the literature encompass: (1) course variables; (2) 
instructor variables; (3) student variables; (4) administration 
variables; and (5) instrument of measurement variables. Student bias 
about faculty gender has been found to affect evaluation of 
instruction. Students appear to notice gender communication pattern 
differences r and these differences are related to specific evaluation 
items. Differences in socialization affect student evaluation of 
instruction; sociocultural stereotyping is a field requiring a great 
deal of study. The implications for future research are: teacher 
evaluation instruments should be analyzed for genaer and race 
differences in communication patterns; general university-wide ^ or 
faculty-wider norms should not be established; a useful evaluation 
system should include common contextual variable norms; and research 
is needed in sociocultural contexts to ensure that instruments for 
measuring student evaluation of instruuLion are not based on 
stereotypes. Five tables summarize the discussion. A 49-item list of 
references is included. (SLD) 
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INTERPRETATION OF STUDENT DATA: CONTEXTUAL AND 
SOCIOCULTURAL VARIABLES 

Speech communication theories share a similarity 
with the interpretation of student evaluation of 
instruction contextual variables. The field of speech 
communication has identified two types of theories: a 
general theory of human communication and style specific 
theory (Baker, 1984; Bormann, 1980). The general theory 
of human communication is a set of generalizations about 
communication that hold regardless of any specific 
context. The style specific theory is based on the 
assumption that communication styles may differ: among 
specific groups of people, common language usage reflects 
cultural values, different verbal and nonverbal 
communication norms, stereotyping and prejudice. 

The similarity of research Is found with 
interpretation of student evaluation of instruction when 
the question is asked, "Can generalizations about teaching 
effectlvoness as evaluated by student ratings of 
instruction hold regardless of any specific context 



variable?" Meta-analysis, a quantitative method used to 
convert individual study outcomes to a common 
denominator, has attempted to determine what context 
variables may effect student ratings of teacher 
effectiveness characteristics. 

The presentation today will explore the common 
elements identified as context variables that may effect 
student evaluation of instruction and present literature 
from sociology, anthropology, and linguistics, that seeks 
renewed challenges to researchers in this area. The.'^e 1 
have called the sociocultural context variables which are: 
gender, race, ethnicity, and social class variables. The 
Sociocultural context variables will be discussed in terms 
of stereotyping and prejudice found with non-dcminant 
groups. 

According to Braskamp, Brandenburg & Ory (1984) 
students are viewed as appropriate sources of describing 
and judging, such as: 1. student-instructor relationships, 
2. their views of the instructors' professional and ethical 
behavior, 3. their workload, 4. what they have learned in 
the course, 5. fairness of grading, and 6. the instructors' 



ability lo communicate cleariy. Dimensions of 
commonality (Brandenburg, Derry & Hengstler, 1978; 
Centra,1980; Doyle, 1983; Mclteaciiie, 1979) in 
questionnaire itema liave identified the following areas: 
communication sitill, rapport with students, course 
organization, student self-rated accomplishments, course 
difficulty, and grading and examinations. 

Context variables identified with affecting student 
evaluation of instruction are numerous and somewhat 
inconsistent. Discrepancy in conclusions according to 
Abrami, Cohen & d'Apollonia (1988) are partly attributable 
to methodology and interpretation discrepancies. The use 
of meta-analysis versus the traditional narrative review, 
has discrepancies in the interpretation by reviewers of 
collective findings. 

Common context variables as identified in the 
literature (Brady, 1989; Braslcamp, et al, 1984; Cranton & 
Smith, 1986; Hoffman, 1978; Marsh, 1980, 1984; Marsh, 
Overall & Kesler, 1979) encompass the following broad 
dimensions: (See Table 1) 



course v ariables, such as, required/elective, day or 
evening, courfie leve!, lecture v discussion, learning value, 
organization, exams/grades, assignments, worlcioad 
difficulty, class size, and academic discipline; 

Instructor variables, such as, rank, gender, full-time 
V. part-time, individual rapport, years teaching, and 
personality characteristics (I.e., warmth, enthusiasm); 

student variables, such as, full-time v. part-time, 
expected grade, prior interest, academic major v. minor, 
grade point average, group interaction, gender, and 
personality characteristics; 

administration variables . such as, student anonymity, 
direction giving, instructor remains in classroom, when in 
semester evaluations are given, etc.; and, 

Instrument variables, such as, placement of items, 
number of response alternatives, negative wording of 
items, and the labeling of all scale points versus labeling 
only end points. 
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Cr^shin (1988) has stated that the multidimensionaiity 
of factors when discussed in terms of reliability 
(consistency, stability and generalizability) concluded that 
no single rating item or set of related Items can be useful 
for all purposes. 

Now we will explore the communicator aspect. 
According to Norton (1977) teaching effectiveness is a 
function of ones' perceived communicator style. 
Communicatee style Is the way one verbally and 
paraverbally interacts to signal how literal meaning should 
be talcen, interpreted, or understood: dominant, dramatic, 
animated, open, contentious, relaxed, friendly, attentive, 
impression leaving, or precise. 

Babad, Bernieri, and Rosenthal (1989) found that 
students' social class and ethnicity serve as major 
variables In the formation of teachers' expectations for 
students' Intellectual performance. Highly biased teachers 
are those susceptible to stereotypically biasing 
information manifesting a strong self-fulfilling prophecy. 
In their study, biased teachers showed a highly differential 
pattern towards either the Pygmalion effect, which is 
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positive treatment of high expectancy students, or the 
Golom effect, which is negative treatment of low 
expectancy students. Bias towards teachers has also been 
found. 

Gender Stereotyping 

Student bias regarding female faculty has been found 
to affect student evaluations of instruction (Basow & 
Distenfeld, 1985; Basow & Howe, 1987; Bennett, 1982; 
Harris, 1976; Kaschak, 1978, 1981). Researchers have 
found that depending on the methodology, the sex-typing of 
the field discussed, the sex-typed characteristics of the 
instructors, and the types of questions asked, female 
professors sometimes receive lower ratings than male 
professors, especially from male students. 

Martin (1984) felt that students have ambivalent 
expectations of female faculty. Women are supposed to be 
warm, friendly, supportive, and deferential, yet 
professionals are supposed to be objective, authoritarian, 
and critical. She believed that female faculty have to walk 
a fine line between feminine warmth and masculine 
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professionalism: straying on either side might resuit in 
lower ratings. 

Bennett (1982) found that women are more negatively 
evaluated than men if they fail to meet gender-appropriate 
expectations with regard to student contact and support. 
Among the personal characteristics associated with high 
ratings for female but not male instructors were 
confidence and decisiveness. It seemed to Martin (1984) 
that female faculty must try harder than their male 
colleagues to convince students that they are both well 
prepared and lilceable. Similarly, Bennetts' data (1982) 
revealed students are less tolerant of female instructors 
in a number of respects, expecting more of them than their 
male colleagues in both educational and interpersonal 
aspects of teaching. 

Wheeless & Potori (1989), and Schein (1975) 
postulated that sex role congruency hypothesis advocates 
behavior that is consistent with sex role stereotypes 
produces more positive outcomes than behavior that is 
inconsistent. Hence, when men and women communicate 
and behave in ways congruent with societal definitions of 
dominant/masculine and expressive/feminine behaviors, 



positive outcomes wiil occuir. MasculSlne qualities of 
teaching effectiveness are decisiveness and deiiberateness 
(Kearney & McCroslcey, 1980) and the maintaining of control 
(Deshpande, Webb & Marks, 1970). Feminine qualities of 
teaching effectiveness are immediacy (Andersen, 1979) non 
dominant behavior (Norton, 1977), and caring and 
understanding (Rubin, 1981). 

A highly structured instructional approach as 
described by students as communicating greater 
professionalism was found to be consistently mor'S 
important for females' performance ratings than for males 
(Bennett, 1982). This was especially true for students' 
ratings of instructors organization, clarity and coherence 
in classroom presentation, command of material for 
classroom presentation, and overall evaluation. Bennett 
(1982) found that students are clearly more tolerant of 
what they perceive as a lack of formal professionalism in 
the conduct of teaching from male professors, demanding 
of women a higher standard of formal preparation and 
organization. Research (Harris, 1976) on sex-role 
stereotyping has suggested that both males and females 

er|c ^ 



tend to perceive male stereotyped behaviors as superior to 
and healthier than female-stereotyped behaviors. 

Sufficient research and documentation (Baird & 
Bradley, 1979; Kahn, 1984; Kotthoff, 1984; Shalceshaft, 
1987) has defined perceived differences and behavioral 
differences pertaining to gender patterns of 
communication. This documentation show that two distinct 
language patterns exist, one for women and one for men. 
(See Tables 2 and 3) Examples of these distinct 
language patterns are: 

Female Patterns Male Patterns 

Correct speech forms More frequent use of 



joking 



Dynamic intonations, wide 
range of pitch 



Hostile verbs 



Polite, cheerful intonatio 



Interir'uptions in 
conversation with 
females 



Use of expressive intensifiers 



Greater amount of 
talicing 



Use of questions to express 
opinions 



Lower pitch levels 
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In two studies by Papalewis & Brown (1989, 1990) of 

35 faculty (17 male and 18 female), with 729 students 

(240 male and 499 female), across 30 different graduate 

and undergraduate classes in Education, the following 

results (see Table 4) were found: In terms of 

communication patterns used by university faculty, as 

predicted from the literature, significant differences were 

found between female and male faculty, such that. 

Females, more than males, were rated 

significantly higher on items regarding: 

*used personal examples in elaborating 
course content, 

*were emotionally involved at times with 
course topics 

*used a wide range of rate and pitch when 
speaking 

^pointed out content areas of personal 
uncertainty 

In terms of student evaluation of instruction, significant 
differences were found between male and female faculty, 
such that: 

Males, more than females, were rated 

significantly higher on items regarding: 

^course clarity, 

^stating objectives, and 
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^organization 

Females, more than males, were rated 

significantly higher on items regarding: 

^enthusiasm, and 

^interest. 

In terms of relationshi ps between aender communication 

patterns and student evaluation of Instruction, small to 

moderate positive correlations were found between: 

Mhe use of personal examples and all evaluation 
Items, 

^emotional involvement and items reflecting 
enthusiasm and interest, and 

*the use of wide range of rate and pitch, when 
speaking with items concerning enthusiasm 
and Interest. 

An Inverse relationship was found between the pointing out 
of areas of personal uncertainty and items regarding 
course clarity, stating of objectives, and organization. 

Papalewis and Brown (1989, 1990) concluded that 
students do notice gender communication pattern 
differences and such differences are related to specific 
evaluation items. 
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Sociocultural Stereotyping 

As was found with gender, Intercultural factors 
affecting interaction and outcomes of conversation based 
on cultural differences requires focused research 
attention. Differences in socialization not only affect how 
others perceive and behave, but also how ones' own 
perception and behavior may contribute to others 
perception and behavior. 

Schnell (1987) defined commurJcation as the 
transmission of information, ideas, attitudes, or emotions 
from one person to another, by conveying those Ideas 
through written or spoken symbols or other verbal or 
nonverbal signs. Perceptions, Schnell (1987) stated, are 
information based on inputs from our senses and 
interpretation of this data on past and present experiences. 
Hence, verbal and nonverbal symbols influence Interracial, 
interpersonal, and intercultural communication in negative 
and positive ways. Some of the negative effects include 
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racist language and excluding behavior. Prejudice often 
exemplifies itself within these negative effects. 

As cited by Booth-Butterfield & Jordan (1989) other 
researchers have identified these race differences: 
African Americans and whites differed In: how they 
reduced uncertainty in initial interactions; the question 
asking pattern and talk time varied by race and gender; 
African American communicators appear to use eye contact 
differently than do white communicators; African American 
communicators exhibit more assertive behavior; and, 
African American communicators displayed a heightened 
emotional expressiveness. Hence, African American 
communicator style is characterized as more outgoing, 
assertive, and expressive than white communicator style. 

Foeman and Pressiey (1987) found that African 
Americans tend to confront individuals immediately when 
dealing with conflict. As cited by Foeman and Pressiey 
(1987), Royce (1982) stated that an African American must 
fairly consistently deal with stereotypes of African 
Americans, and whites, though they tend to think of each 
other as individuals, think of blacks in more sweeping 
terms. 



Booth-Butterfield and Jordan (1989) stated that in 
order to elicit favorable evaluations, African American 
communicators often find they need to emulate white 
communication style in educational settings. Citing 
Rubins* study (1986), African American communication 
effectiveness suggested that adaptation to predominantly 
white standards must occur for African Americans to be 
perceived as effective communicators in racially 
heterogeneous classrooms. 

Even less cross-cultural research has been done with 
Hispanics, Native Americans, and Asian Americans. One 
study done by Meredith (1988) found that Japanese 
Americans and female students preferred a more "female" 
style of teaching, where the teacher exhibited openness, 
warmth, respect, and accessibility. 

Aclcnowledging the complexity of contextual variables, 
and the implications of sociocultural variables, should lead 
us in the direction of future research. The sociocultural 
variables present another dimension required to accurately 
assess student evaluation of instruction. The macro 
approach of generalizabiiity of findings in this research 
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area, needs to be balanced with a micro approach: untii 
stereotypic dimensions inherent In human behavior are 

explored, we will continue to have research reflect the 
dominant culture. 

Implications 

As stated by Baker (1984) to avoid the trappings of 
ethnocentric bias, researchers must begin to focus on 
cultural values, different verbal and nonverbal 
communication norms, and stereotyping. Implications for 
future research are: (1) Teacher evaluation instruments 
in practice should be analyzed for gender and race 
differences in communication patterns; (2) General 
university-wide norms or even faculty-wide norms in a 
particular college/school or multi-program department, 
should not be established; (3) Possibly, the most useful 
evaluation system will include common contextual variable 
norms, and; (4) More research is needed in sociocultural 
contexts to ensure student evaluation of instruction 
instruments are not stereotypic biased. 
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TABLE 1 DIMENSIONS OF CONTEXT VARIABLES 



COURSE VARIABLES 
Required/elective, Day or evening, Course level, Lecture v 
discussion. Learning value. Organization, Exams/grades, 
Assignments, Workload difficulty. Class size, and 
Academic discipline 

INSTRUCTOR VARIABLES 
Rank, Gender, Full-time v. part-time. Individual rapport. 
Years teaching, and Personality characteristics (i.e., 
warmth, enthusiasm) 

STUDENT VARIABLES 
Full-time V. part-time. Expected grade. Prior interest, 
Academic major v. minor. Grade point average. Group 
interaction. Gender, and Personality characteristics 

ADMINISTRATION VARIABLES 
Student anonymity. Direction giving. Instructor remains in 
classroom, When in semester evaluations are given, etc. 

INSTRUMENT VARIABLES 
Placement of items, Number of response alternatives, 
Negative wording of items, and the Labeling of all scale 
points versus labeling only end points. 
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TABLE 2 



GENDER PATTERNS OF COMMUNICATION 



Male 

More frequent use of 
joking 

Shakeshaft,1987, 

Aggressive Speecti 
Scott, 1979 



Interruptions in 
conversations with 
females 
Scott, 1979 



Greater amounts of 

talking 

Scott, 1979 



Lower pitch levels 
Scott, 1979 



Patterns 

Demanding voice 
Deep voice 
Boastful speech 
Quantifying modifiers 
Use swear words 
Dominating speech 
Loud speech 
Show anger 
Straight to the point 
Militant speech 
Hostile verbs 
Use slang 

Authoritarian speech 
Forceful speech 
Lean back while talking 
Aggressive speech 
Blunt speech 
Interrupt women 
Humor in speech 
Passive voice 
Prefers impersonal 
Generalizes, 3rd person 
Evidence of fact rather 
than feeling 
Scott, 1979; 
Shakeshaft, 1987 
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TABLE 3 



GENDER PATTERNS OF COMMUNICATION 



Female Patterns 



Correct speech forms 
Scott, 1979 



Dynamic intonations, 
wide range of pitch 
Shalceshaft, 1987 



Polite, cheerful 
intonation 
Scott, 1979 



Use of expressive 
intensifiers 

Scott, 1979; Lakoff, 1975; 



Use of questions to 
express opinions 
Lakoff, 1975: Shakeshaft, 
1987 



Enunciate clearly 
High pitch 

Use hands and face to express 

ideas 

Gossip 

Concern for listener 

Gentle speech 

Fast speech 

Use of intensifiers 

Talk about trivial topics 

Wide range of rate and pitch 

Friendly speech 

Talk a lot 

Emotional speech 

Use many details 

Smooth speech 

Open, self-revealing speech 

Enthusiastic speech 

Smile a lot when talking 

Good grammar 

Polite speech 

Question intonation 

Tag questions 

Use of qualifiers 

Superlatives, diminutives 

Scott, 1979; Shakeshaft, 

1987 
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TABLE 4 COMMUNICATION PATTERNS AND STUDENT 

EVALUATION OF INSTRUCTION 



Communication Pattern Differences 

FemaleSt more than males, were rated significantly higher on 
Items regarding: 

Used personal examples In elaborating course content, 
Were emotionally Involved dt times with course topics 
Used a wide range of rate and pitch when speaking 
Pointed out content area? of personal uncertainty 

Student Evaluation Of Instruction Differences 

Males, more than females, were rated significantly higher on 
Items regarding: 

Course clarity. 

Stating objectives, and 

Organization 

Females, more than males, were rated significantly higher on 
Items regarding: 

Enthusiasm, and 

interest. 

Relationship Between Gender Communication Patterns And Student 
Evaluation Of Instruction small to moderate positive correlations 
were found between: 

The use of personal examples and all evaluation items. 

Emotional involvement and Items reflecting enthusiasm and 
interest, and 

The use of wide range of rate and pitch, when speaking 

with items concerning enthusiasm and interest. 
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TABLE 5 



AFRICAN AMERICAN PATTERNS 
OF COMMUNICATION 



Confront individuals immediately Foeman & Pressley, 

1987 



Tentative Booth-Butterfield & 

Jordan, 1989 



Use Eye Contact Aiello & Jones, 

1970; Eril(son, 
1979, LaFrance & 
Mayo, 1976 

Higti Emotional Expressiveness 

Foeman & Pressley, 
1987; Gumperz & 
Tannen, 1979 

Assertiveness Foeman & Pressley, 

1987 
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